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ABSTRACT 



To help teachers, administrators, and parents understand 
student performance in writing, a state- level report is prepared each year to 
analyze students writing scores and provide guidelines for the interpretation 
of the results. This report compares students scores on the 2001 Delaware 
Student Testing Program (DSTP) with scores on the 2000 DSTP using the new 
scoring rules for writing. In 2001, Delaware students in grades 3, 5, 8, and 
10 took the DSTP writing assessment. Each student responded to a text-based 
writing task and a stand-alone writing prompt. Each students response to the 
text-based task was scored by one trained reader using a five-point scoring 
rubric, and responses to the stand-alone writing were scored by two raters. 
This report contains the newly calculated means and standard deviations for 
the writing scores from 1998 through 2000. The generalizability of writing 
scores is low across the discourses of writing tasks, writing topics, and 
occasions, but a summary is presented for reference uses. The results of the 
writing assessments suggest that writing performance remained similar to the 
previous year in grades 3 and 10, but improved slightly for grades 5 and 8. 
For the stand-alone task, there was no difference from the previous year for 
grades 3 and 10, but slightly higher scores for grades 5 and 8. A similar 
pattern was seen for the text-based writing task. A cross-grade pattern shows 
that the frequency distributions of writing scores shifted gradually from the 
lower- score end to the higher- score end, so that more students received a 
higher score, and fewer a lower score, in 2001 than in 2000. An attachment 
contains supplemental information about test results. (SLD) 
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State Report of the 2001 DSTP Writing Assessment 



I. Introduction: 

To help teachers, administrators, and parents understand student performance in writing, 
the Assessment and Analysis Group prepares a state- level report each year. We analyze 
students’ writing scores and provide guidelines for the interpretation of the results. In 
this report, we compare students’ scores on the 2001 DSTP with students’ scores on the 

2000 DSTP using the new scoring rules for writing (For detailed information, please read 
the section of DSTP Writing Assessment). 

II. DSTP Writing Assessment 

In 2001, Delaware students in grades 3, 5, 8, and 10 took the DSTP writing assessment. 
Each student responded to a text-based writing task and a stand-alone writing prompt. 
The text-based writing task was linked to a passage in the DSTP reading test. For the 
stand-alone writing task, while students were encouraged to use prewriting skills to 
develop, organize, and draft their responses to the prompt. Only the final draft was 
scored. 

Each student's response to the text-based writing was scored by one trained reader using a 
5-point scoring rubric; responses to the stand-alone writing were scored by two trained 
readers using the same scoring rubric (See Attachment A) and the sum of the two scores 
was reported. The bwest score for the text-based writing is 0 and the highest possible 
score is 5; the lowest score for the stand-alone writing is 0 and the highest possible score 
is 10. The total writing raw score is a combination of the text-based writing score and the 
stand-alone writing score with the lowest score of 1 and the highest possible score of 15. 
According to the new scoring rule, if a student receives a 0-score on one writing task, but 
a valid, non 0-score on the other writing, this student will receive a valid total writing 
score. However, if a student misses both parts of the writing assessment, such as a 0- 
score, an invalid score, this student will not receive a valid total writing score. The newly 
calculated means and standard deviations for the 1998, 1999, 2000, and 2001 writing 
scores are presented in this report, however, interpretations and comparisons were 
focused on the 2000 and 2001 data. 

III. Results of DSTP Writing Assessment 

Overall Writing Performance To make a more meaningful comparison of writing scores 
across years, effect size is used for analysis. The measure of effect size (ES) or standard 
mean difference varies from zero. That is, when there is no difference, ES is zero; if the 

2001 average score is higher, ES is positive; if the 2000 average score is higher, ES is 
negative. Effect size is comparable from measure to measure, even if the score scales on 
the measures are different. A widely used criterion for the interpretation of ES -values as 
proposed by Cohen (1988), evaluates .20 - .49 as a small difference, .50 - .79 as medium, 
and .80 or higher as large. Many researchers, however, strongly encourage using these 
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criteria as a reference only and consider the particular situation of a given test to avoid 
misleading interpretations (Willingham and Cole, 1997; Glass et al, 1981). In this report, 
ES-values are interpreted in the latter way. 

Tables la and lb present the types of writing and descriptive statistics (means and 
standard deviations) for the 2001 and 2000 writing assessments using the new scoring 
rules. The average writing scores are 5.89 for grade 3, 7.34 for grade 5, 7.92 for grade 8, 
and 7.38 for grade 10 in 2001. The results of effect size suggest that (1) all ES-values are 
positive ranging from .01 to .44; (2) the near zero ES for grade 3 (ES=.01) and 10 
(ES=.17) indicate no significant difference in the average writing scores between 2001 
and 2000; and (3) the small ES for grades 5 (ES=.40) and 8 (ES=.44) indicate a slightly 
higher average score for 200 1 than 2000. 

Further review of students’ writing scores (Table 2a- 2b) found a cross-grade pattern: the 
frequency distributions of writing scores shifted slightly from the lower end to the higher 
end in 2001 compared with 2000. In grade 3, the percentage of students having 6-points 
increased from 22% in 2000 to 29% in 2001; whereas the percentage of students having 
5-points dropped from 25% in 2000 to 20% in 2001. In grade 5, the percentages of 
students having 8- and 9-points increased from 10% and 19% in 2000 to 17% and 23%, 
respectively, in 2001; whereas the percentages of students having 5-, 6-, and 7-points 
dropped from 17%, 18%, and 20% in 2000 to 10%, 15%, and 17%, respectively, in 2001. 
In grade 8, the percentages of students receiving 9- and 10-points increased from 12% 
and 4% in 2000 to 27% and 9%, respectively; whereas, the percentages of students 
receiving 5-, 6-, 7-, and 8-points dropped from 1 1%, 14%, 22%, and 30% in 2000 to 4%, 
11%, 14%, and 28%, respectively. In grade 10, the percentage of students having 8- 
points increased from 22% in 2000 to 34% in 200 1 ; whereas the percentages of students 
having 5- and 6-points dropped from 13% and 16% in 2000 to 6% and 12%, respectively, 
in 2001. 

The percentage of students at each performance level is nearly the same in 2000 and in 
2001 for grade 3 (Tables 3a-3b). In grade 5, the percentage increased by 16% for 
meeting the standard, and 2% for exceeding the standard from 2000 to 2001; the 
percentage dropped by 12% and 6% for well below and below the standard, respectively. 
In grade 8, the percentage increased by 17% and 2% for meeting and exceeding the 
standard, respectively; the percentage dropped by 8% and 1 1% for well below and below 
the standard, respectively. Similarly, in grade 10, the percentage increased by 14% for 
meeting the standard from 2000 to 2001; the percentages for well below and below the 
standard dropped by 8% and 4%, respectively. 

Performance on Stand-Alone Writing The average score of stand-alone writing (Table 
la- lb) is slightly bwer for grade 3 in 2001 than 2000 (4.27 vs. 4.40), but slightly higher 
for grade 5 (5.26 vs. 4.90), 8 (5.56 vs. 5.49), and 10 (5.50 vs. 5.29). The near-zero ES for 
grade 3 (-.10), 8 (.06), and 10 (.16) indicates that there is no significant difference in the 
stand-alone score over time. An ES of .26 in grade 5 suggests a slight increase of the 
average performance from 2000 to 2001. 
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The frequency distributions of stand-alone scores show varying patterns from grade to 
grade. In grade 3, the percentage ofO-, 5-, and 6-points dropped by 2%, 3%, and 5%, 
respectively, from 2000 to 2001; whereas the percentage of 4-points increased by nearly 
12%. In grade 5, the percentage of students having 5-points or below dropped by 1% to 
5%; whereas the percentage of students having 6- to 8-points increased by 2% to 4%. In 
grade 8, the percentage of students receiving 6-points increased by 7%, but the 
percentage of 4- and 5-points dropped by 2% to 3% from 2000 to 2001. In grade 10, the 
percentage of students having 6- and 7-points increased by 3% to 7%; whereas the 
percentage of 4- and 5-points dropped by 3% to 10% across years. 

Performance on Text-Based Writing The average score for text-based writing (Table la- 
lb) is slightly higher in 2001 than 2000 for grades 3 (1.66 vs. 1.50) and 10 (2.02 vs. 1.94) 
and higher for grades 5 (2.12 vs. 1.70) and 8 (2.44 vs. 1.80). The near- zero ES for grades 
3 and 10 indicates no significant difference in text-based scores across years; the ES of 
.54 for grade 5 suggests that the 2001 average score is about a half of standard deviation 
higher than the 2000 average score; the ES of .89 for grade 8 suggests that the 2001 
average score is nearly one standard deviation higher than the 2000 average score. 

Table 5 shows that a similar jattem across grades that the shape of the frequency 
distributions seemed to shift from lower scores to higher scores for text-based writing. In 
grade 3, the percentage of 2points increased from 35% in 2000 to 53% in 2001, 
respectively; but the percentage of 1-point dropped from 57% to 30%, respectively. In 
grade 5, the percentages of 2- and 3-points increased from 40% and 13% in 2000 to 49% 
and 29% in 2001, respectively; but the percentages of 0- and 1-point dropped from 3% 
and 41% to less than 1% and 20%, respectively. In grade 8, the percentage of 3- and 4- 
points increased from 12% and 1% in 2000 to 37% and 8% in 2001, respectively; but the 
percentage of 1- and 2-points dropped from 32% and 55% in 2000 to 8% and 47% in 
2001, respectively. Similarly in grade 10, the percentage of 2-points increased from 45% 
in 2000 to 69% in 2001; but the percentage of 1-point dropped from 32% to 14%. The 
percentage of 3- and 4-points dropped slightly from 18% in 2000 to 16% in 2001 and 
from 4% to less than 1%. 

It is important to note that 630 third graders received a 0-score on the text-based writing 
in 2001. Further analysis was conducted to determine if geographic location (by district 
and school), student background (by special education and limited English proficient), 
and writing skills (their scores on the stand-alone writing) were the possible reasons for 
0-scores (Attachment B). No particular patterns have been found. Since then the 
Assessment and Analysis Group, the English language arts Test Devebpment 
Committee, and the DSTP Technical Advisory Committee have reviewed the text-based 
writing data and made recommendations to improve the text-based writing. A research 
study is currently under design for collecting additional information to support the final 
decisions. 
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IV. Summary 

It is very important to note that the generalizability of writing scores is low across the 
discourses of writing tasks, writing topics, and occasions. The following summary based 
on the results of statistical analysis can only be used as a reference. Even though effect 
size is used for analysis in this report, caution should be taken when using the results of 
cross-year comparisons. Since only raw scores are reported in writing, comparisons 
across grades are inappropriate. 

1. In 2001, the average writing score was 5.98 for grade 3, 7.34 for grade 5, 7.92 for 
grade 8, and 7.38 for grade 10. The results of analysis suggest that writing performance 
remained similar to the previous year for grades 3 and 10. The 200 l’s scores were 
slightly higher than 2000 for grades 5 and 8. 

2. The average score for the stand-alone writing task was 4.27, 5.26, 5.56, and 5.50 for 
grades 3, 5, 8, and 10, respectively in 2001. The near- zero effect size suggests no 
difference in the stand-alone writing scores from the previous year for grades 3 and 10, 
but slightly higher scores for grades 5 and 8 in 2001. 

3. The average score for the text-based writing task was 1.66, 2.12, 2.44, and 2.02 for 
grades 3, 5, 8, and 10, respectively. The near- zero effect size for grades 3 and 10 
indicates no difference in text-based scores between 2000 and 2001; the effect sizes of 
.54 and .89 for grades 5 and 8, respectively, suggest that the 2001 average score is about 
one half to nearly one standard deviation higher than the 2000 average score. 

4. A cross-grade pattern shows that the frequency distributions of writing scores shifted 
gradually from the lower- score end to the higher- score end. In other words, more students 
received a higher score and fewer students received a lower score in 2001 than in 2000 on 
both stand-alone and text-based writing tasks. 
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Means, Standard Deviations, and Effect Sizes of Writing Scores 
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Table 1 b 

Means and Standard Deviations of Writing Scores 
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Table 2a 

Frequency Distributions of Writing Scores 









2001 




2000 


Grade 


Score 


Frequency 


% 


Frequency 


% 


3 


1 


25 


0.29 


157 


1.99 




2 


212 


2.44 


86 


1.09 




3 


505 


5.81 


463 


5.87 




4 


834 


9.59 


712 


9.03 




5 


1728 


19.86 


1953 


24.77 




6 


2541 


29.21 


1741 


22.08 




7 


1413 


16.24 


1340 


16.99 




8 


951 


10.93 


911 


11.55 




9 


376 


4.32 


359 


4.55 




10 


77 


0.89 


116 


1.47 




11 


24 


0.28 


31 


0.39 




12 


10 


0.11 


12 


0.15 




13 


0 


0.00 


3 


0.04 




14 


2 


0.02 


2 


0.03 




15 


1 


0.01 


0 


0.00 




Total 


8699 


100.00 


7886 


100.00 








2001 




2000 


Grade 


Score 


Frequency 


% 


Frequency 


% 


5 


1 


33 


0.39 


68 


0.90 




2 


48 


0.57 


101 


1.34 




3 


238 


2.83 


272 


3.60 




4 


303 


3.60 


502 


6.64 




5 


801 


9.51 


1248 


16.51 




6 


1264 


15.01 


1396 


18.47 




7 


1461 


17.35 


1480 


19.58 




8 


1912 


22.70 


1406 


18.60 




9 


1460 


17.33 


718 


9.50 




10 


556 


6.60 


250 


3.31 




11 


254 


3.02 


88 


1.16 




12 


73 


0.87 


22 


0.29 




13 


19 


0.23 


5 


0.07 




14 


1 


0.01 


3 


0.04 




15 


0 


0.00 


1 


0.01 




Total 


8423 


100.00 


7560 


100.00 
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Table 2b 

Frequency Distributions of Writing Scores 









2001 




2000 


Grade 


Score 


Frequency 


% 


Frequency 


% 


8 


1 


19 


0.22 


41 


0.50 




2 


40 


0.47 


53 


0.65 




3 


75 


0.88 


59 


0.72 




4 


167 


1.95 


291 


3.56 




5 


354 


4.13 


870 


10.64 




6 


956 


11.16 


1145 


14.00 




7 


1193 


13.93 


1803 


22.05 




8 


2389 


27.89 


2483 


30.37 




9 


2278 


26.59 


947 


11.58 




10 


759 


8.86 


353 


4.32 




11 


249 


2.91 


102 


1.25 




12 


82 


0.96 


22 


0.27 




13 


5 


0.06 


6 


0.07 




14 


1 


0.01 


1 


0.01 




15 


0 


0.00 


1 


0.01 




Total 


8567 


100.00 


8177 


100.00 








2001 




2000 


Grade 


Score 


Frequency 


% 


Frequency 


% 


10 


1 


39 


0.49 


75 


1.01 




2 


134 


1.69 


78 


1.05 


' 


3 


141 


1.78 


148 


1.99 




4 


287 


3.62 


349 


4.69 




5 


466 


5.88 


975 


13.11 




6 


983 


12.40 


1174 


15.79 




7 


1354 


17.08 


1317 


17.71 




8 


2690 


33.93 


1645 


22.13 




9 


1157 


14.60 


970 


13.05 




10 


499 


6.29 


427 


5.74 




11 


156 


1.97 


189 


2.54 




12 


17 


0.21 


76 


1.02 




13 


4 


0.05 


9 


0.12 




14 


0 


0.00 


2 


0.03 




15 


0 


0.00 


1 


0.01 




Total 


7927 


100.00 


7435 


100.00 
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Table 3a 

Frequency Distributions of Performance Levels 
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Table 3b 

Frequency Distributions of Performance Levels 
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Total 7255 100.00 7357 100.00 



Table 4a 

Frequency Distributions of Stand-Alone Writing Scores 









2001 




2000 


Grade 


Score 


Frequency 


% 


Frequency 


% 


3 


0 


27 


0.31 


202 


2.56 




2 


620 


7.13 


534 


6.77 




3 


876 


10.08 


792 


10.04 




4 


4174 


48.03 


2806 


35.55 




5 


1687 


19.41 


1744 


22.10 




6 


1113 


12.81 


1453 


18.41 




7 


156 


1.80 


264 


3.35 




8 


30 


0.35 


88 


1.12 




9 


4 


0.05 


5 


0.06 




10 


3 


0.03 


4 


0.05 




Total 


8690 


100.00 


7892 


100.00 








2001 




2000 


Grade 


Score 


Frequency 


% 


Frequency 


% 


5 


0 


37 


0.44 


80 


1.06 




2 


306 


3.64 


334 


4.42 




3 


331 


3.94 


507 


6.71 




4 


1878 


22.35 


2026 


26.83 




5 


1646 


19.59 


1587 


21.01 




6 


3057 


36.38 


2479 


32.83 




7 


766 


9.11 


401 


5.31 




8 


327 


3.89 


122 


1.62 




9 


50 


0.59 


6 


0.08 




10 


6 


0.07 


10 


0.13 




Total 


8404 


100.00 


7552 


100.00 
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Table 4b 

Frequency Distributions of Stand-Alone Writing Scores 









2001 




2000 


Grade 


Score 


Frequency 


% 


Frequency 


% 


8 


0 


9 


0.11 


25 


0.31 




2 


85 


1.00 


61 


0.75 




3 


158 


1.85 


167 


2.05 




4 


1259 


14.76 


1384 


17.01 




5 


1547 


18.14 


1732 


21.29 




6 


4529 


53.09 


3766 


46.29 




7 


736 


8.63 


769 


9.45 




8 


198 


2.32 


220 


2.70 




9 


8 


0.09 


5 


0.06 




10 


1 


0.01 


7 


0.09 




Total 


8530 


100.00 


8136 


100.00 








2001 




2000 


Grade 


Score 


Frequency 


% 


Frequency 


% 


10 


0 


31 


0.40 


27 


0.37 




2 


208 


2.65 


150 


2.04 




3 


257 


3.28 


227 


3.09 




4 


1132 


14.44 


1741 


23.71 




5 


1478 


18.85 


1530 


20.84 




6 


3469 


44.25 


2751 


37.46 




7 


902 


11.51 


633 


8.62 




8 


352 


4.49 


268 


3.65 




9 


8 


0.10 


10 


0.14 




10 


3 


0.04 


6 


0.08 




Total 


7840 


100.00 


7343 


100.00 





Table 5 

Frequency Distributions of Text-Based Writing Scores 









2001 




2000 


Grade 


Score 


Frequency 


% 


Frequency 


% 


3 


0 


630 


7.38 


53 


0.68 




1 


2549 


29.84 


4434 


57.10 




2 


4487 


52.53 


2677 


34.47 




3 


835 


9.78 


553 


7.12 




4 


37 


0.43 


47 


0.61 




5 


3 


0.04 


2 


0.03 




Total 


8541 


100.00 


7766 


100.00 








2001 




2000 


Grade 


Score 


Frequency 


% 


Frequency 


% 


5 


0 


18 


0.22 


233 


3.15 




1 


1647 


19.86 


3056 


41.27 




2 


4086 


49.26 


2977 


40.21 




3 


2372 


28.60 


975 


13.17 




4 


169 


2.04 


155 


2.09 




5 


2 


0.02 


8 


0.11 




Total 


8294 


100.00 


7404 


100.00 








2001 




2000 


Grade 


Score 


Frequency 


% 


Frequency 


% 


8 


0 


29 


0.35 


63 


0.81 




1 


664 


7.92 


2492 


31.90 




2 


3927 


46.87 


4278 


54.75 




3 


3107 


37.08 


903 


11.56 




4 


651 


7.77 


76 


0.97 




5 


1 


0.01 


1 


0.01 




Total 


8379 


100.00 


7813 


100.00 








2001 




2000 


Grade 


Score 


Frequency 


% 


Frequency 


% 


10 


0 


46 


0.61 


30 


0.43 




1 


1048 


13.81 


2254 


32.00 




2 


5233 


68.96 


3162 


44.90 




3 


1233 


16.25 


1282 


18.20 




4 


25 


0.33 


305 


4.33 




5 


3 


0.04 


10 


0.14 




Total 


7588 


100.00 


7043 


100.00 





Attachment A 



Writing Scoring Rubric 



Delaware Student Testing Program - General Rubric for Writing 
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Attachment B 



Distributions of Off-Topic Responses to the Text-Based 
Writing Task in Grade 3 



Frequency Distributions of 0-Scores on the Text-Based Writing 
Task by District Code in Grade 3 



District Code 


N. 


% 


10 


38 


6.03 


13 


30 


4.76 


15 


25 


3.97 


16 


18 


2.86 


17 


11 


1.75 


18 


20 


3.17 


23 


14 


2.22 


24 


16 


2.54 


29 


20 


3.17 


31 


52 


8.25 


32 


112 


17.78 


33 


87 


13.81 


34 


93 


14.76 


35 


21 


3.33 


36 


39 


6.19 


74 


3 


0.48 


76 


18 


2.86 


84 


13 


2.06 


Total 


630 


100.00 
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Frequency Distributions of 0-Scores on the Text-Based Writing 
Task by School Code in Grade 3 
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Frequency Distributions of 0-Scores on the Text-Based Writing 
Task by Their Total Writing Scores in Grade 3 



Writing Score 


N. 


% 


2 


150 


23.81 


3 


124 


19.68 


4 


278 


44.13 


5 


57 


9.05 


6 


19 


3.02 


7 


2 


0.32 


Total 


630 


100.00 
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Frequency Distributions of 0-Scores on the Text-Based Writing Task by 
Special Education, LEP, and Free/Reduced Price Lunch in Grade 3 
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